feat(chameleon): Chameleon-7B Thor FP8 and Orin SM87 INT8/QuaRot frontends - #166

Thank you for contributing the complete Chameleon-7B Thor FP8 and Orin SM87 frontends. The responsibilities of the model layer, pipelines, and hardware backends are generally separated clearly, and we did not find an existing model being redirected to the Chameleon runtime. Codex reviewed this PR against FlashRT's long-term maintenance standards. We recommend addressing the following items after #165 has been corrected and merged.

Required changes

1. Resolve the dependency and rebase first

   This PR currently contains all commits from feat(csrc): Chameleon-7B kernel layer — fused quant, INT8 GEMMs, FA2 causal attention #165 and conflicts with the latest main in flash_rt/api.py. Please merge feat(csrc): Chameleon-7B kernel layer — fused quant, INT8 GEMMs, FA2 causal attention #165 after its model-level build isolation is complete, then rebase this PR onto the new main, remove the duplicated kernel commits, and resolve the API registry conflict.

2. Fix the Thor prompt-padding capacity boundary

   set_prompt() currently checks the unpadded prompt length against max_seq and pads it to a multiple of 16 afterward. If max_seq is not a multiple of 16, the padded sequence can exceed the allocated buffers and KV-cache capacity.

   Please compute and validate padded_len <= self._Se_max, or normalize the effective capacity during initialization. Add boundary tests covering non-aligned max_seq values and prompts close to capacity.

3. Add an SM110 fail-fast check to the Thor frontend

   The Orin frontend explicitly validates SM87, but the Thor frontend has no corresponding capability check. Please call torch.cuda.get_device_capability() before loading the checkpoint, allocating large buffers, or loading hardware-specific libraries, and report a clear error on non-SM110 devices. If a development override is necessary, it should use a documented, Chameleon-specific environment variable.

4. Fix the Orin generation-parameter boundary

   With max_new_tokens <= 0, the current implementation may still run prefill and return one token. Please define the behavior explicitly: reject negative values with ValueError, and return an empty generated result for zero. The sequence-capacity boundary should be covered by tests as well.

5. Complete the license handling for the vendored VQGAN code

   The vendored source headers state that the files are governed by the Chameleon License, while the repository currently contains only its Apache-2.0 root license and does not include the corresponding third-party license or NOTICE. Please verify the actual source and applicable license:

   • If the code comes from Meta Chameleon, include the complete license, attribution, and modification notice, and confirm that its restrictions are compatible with this repository's distribution policy.
   • If the code comes from the original MIT-licensed CompVis implementation, derive the required code from that source and retain its copyright and MIT license attribution.

   The vendored files should not be released until third-party license compatibility is confirmed.

6. Add contract tests proportionate to the PR's scope

   The current new tests mainly inspect two constructor defaults, which is not enough to protect two substantial frontends and pipelines. At minimum, please cover registry and lazy import behavior, optional dependencies, configuration fail-fast behavior, prompt and generation boundaries, missing-backend errors, and basic eager-versus-graph consistency.

Pre-merge checklist

• feat(csrc): Chameleon-7B kernel layer — fused quant, INT8 GEMMs, FA2 causal attention #165 is merged, and this PR is rebased onto the latest main without duplicated kernel commits
• GitHub reports the PR as mergeable, with no flash_rt/api.py conflict
• With Chameleon disabled, existing models retain their default build and runtime behavior
• Thor rejects non-SM110 devices early, and Orin rejects non-SM87 devices early
• Thor padding and max_seq boundary tests pass
• Negative, zero, and capacity-clipped max_new_tokens behavior is tested
• Import, registry, and optional-dependency smoke tests pass
• SM87 and SM110 each pass model-selected build, import, prefill, and decode smoke tests
• Vendored VQGAN licensing and attribution are complete
• git diff --check, Python compilation, and relevant tests pass

Please also refer to the repository's PR Review Checklist and Adding a New Model guides.

This is a Codex-assisted maintainability review. The architecture is moving in a reasonable direction; the dependency order, model build isolation, input boundaries, and third-party distribution requirements need to be completed before these hardware frontends are suitable for long-term support on main.

Thanks for the review. The branch has been rebased onto the corrected #165 head and all six items are addressed in 193797e; once #165 merges, this PR will be rebased onto main with the duplicated kernel commits dropped and the flash_rt/api.py conflict resolved. Item-by-item:

1. Dependency/rebase — rebased onto the corrected #165 head; will rebase onto main immediately after #165 merges.

2. Thor prompt-padding capacity boundary — capacity is now floored to a multiple of 16 at allocation (_allocate_buffers), and set_prompt validates the padded length against that capacity, so a non-aligned max_seq can never let the pad-to-16 overshoot the buffers or KV cache. Covered by tests/test_chameleon_contracts.py: non-aligned-capacity overshoot raises, in-capacity prompts pad correctly, and an exact multiple-of-16 prompt passes without padding.

3. SM110 fail-fast — ChameleonTorchFrontendThor now calls torch.cuda.get_device_capability() before checkpoint loading, buffer allocation, and hardware-specific library loading, raising a clear RuntimeError on non-SM110 devices. The dev override is the documented, Chameleon-specific FLASHRT_CHAMELEON_THOR_FORCE=1. Covered by mocked-CUDA tests (rejects no-CUDA and wrong capability, accepts SM110, honors the override) mirroring the Orin frontend's existing gate.

4. Orin generation-parameter boundary — generate() now defines the behavior explicitly: negative max_new_tokens raises ValueError, zero returns an empty result with no prefill/decode (previously it ran prefill and returned one token), and over-capacity values are clipped with a warning. Thor generate_greedy got the same negative-value rejection. Covered by tests; the sequence-capacity clip warning is exercised by the existing boundary.

5. VQGAN licensing — the vendored files are from Meta Chameleon (chameleon/vae/), whose core vqgan.py derives from the MIT-licensed CompVis taming-transformers. I've added the complete Chameleon Research License text (flash_rt/models/chameleon/vqgan/LICENSE) and a NOTICE with provenance (including the preserved CompVis MIT attribution), the inference-only modification record, and a compatibility notice. One point needs a maintainer policy decision: the Chameleon Research License is noncommercial-research-only, which is more restrictive than this repository's Apache-2.0. I have not re-derived the files from the MIT CompVis source because that would alter production-validated code. If the project's distribution policy can't accept a noncommercial component, the options are (a) re-derive from the MIT CompVis implementation, or (b) substitute an independently-licensed VQ-GAN. I did not remove the files pending that decision; please advise.

6. Contract tests — tests/test_chameleon_contracts.py adds: registry map + lazy-module-string checks, resolve_pipeline_class resolution for both hardware targets, the SM87 allowlist rejecting unsupported configs, the load_model(config="chameleon") redirect, both hardware fail-fast gates, prompt padding bounds, and generation-parameter bounds. Eager-versus-graph consistency is covered by the existing precision harness (scripts/check_chameleon_thor_precision.py) which requires a real checkpoint; happy to convert it to a checkpoint-gated pytest gate if you prefer it in-tree.

Production Orin re-test of this branch (build matrix + contract tests) is queued; results will be reported here.

Production re-test of 193797e on Jetson Orin (sm_87, CUDA 12.2, torch 2.3.0) — all green:

• Build/symbol matrix correct after rebase onto the corrected feat(csrc): Chameleon-7B kernel layer — fused quant, INT8 GEMMs, FA2 causal attention #165 (Chameleon OFF default surface unchanged; ON build/import/symbol checks pass).
• tests/test_chameleon_contracts.py passes: registry/lazy-import, load_model(config="chameleon") redirect, SM87 allowlist, Orin hardware fail-fast and env override, prompt pad-to-16 capacity boundaries (non-aligned overshoot rejection), and max_new_tokens negative/zero behavior.
• 0 failures; all skips are documented hardware-capability gates (Thor-only kernels, sm_89+ FP8).

Outstanding: the VQGAN license policy decision flagged above, and the #165 merge + rebase onto main.